Papers with RAG models
CLAPnq: Cohesive Long-form Answers from Passages in Natural Questions for RAG systems (2025.tacl-1)
Copied to clipboard
| Challenge: | Retrieval Augmented Generation (RAG) is a popular application for large language models. |
| Approach: | They present a benchmark Long-form Question Answering dataset for RAG . they use a passage grounded in a gold passage to provide accurate answers . |
| Outcome: | The proposed dataset provides long answers with grounded gold passages from Natural Questions (NQ) the answers are concise, 3x smaller than the full passage, and cohesive, meaning the answer is composed fluently. |
Retrieval-augmented Generation across Heterogeneous Knowledge (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for retrieving knowledge from a single source homogeneous corpus have been gaining increasing attention in the field of natural language processing (NLP) however, they still suffer from the following drawbacks: (i) They are usually trained offline, making the model agnostic to the latest information, e.g., asking a chat-bot about COVID-19. |
| Approach: | They propose to use a single-source homogeneous corpus to generate retrieval-augmented generation models that can learn from the pre-training corpus. |
| Outcome: | The proposed methods have been applied to various knowledge-intensive NLP tasks, but most of the work has focused on retrieving unstructured text documents from Wikipedia. |
CoRAG: Collaborative Retrieval-Augmented Generation (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing research on Retrieval-Augmented Generation models has focused on centralized settings where a single entity controls both the model and the datastore. |
| Approach: | They propose a framework for RAG where clients jointly train a shared model using a collaborative passage store. |
| Outcome: | The proposed framework outperforms parametric learning methods and locally trained models in low-resource scenarios. |
Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on retrieval-augmented generation (RAG) rarely address the issue of predictive uncertainty, i.e., how likely it is that a RAG model’s prediction is incorrect. |
| Approach: | They propose a framework that induces RAG models to alter latent factors and analyzes the effect on their answers. |
| Outcome: | The proposed framework identifies two critical factors affecting RAG models' confidence in their answers and analyzes the effect on their answers. |
Intuitive or Dependent? Investigating LLMs’ Behavior Style to Conflicting Prompts (2024.acl-long)
Copied to clipboard
| Challenge: | Extensive experiments with seven Large Language Models reveal their varying behaviors. |
| Approach: | They investigate the behaviors of Large Language Models when faced with conflicting prompts versus their internal memory. |
| Outcome: | Extensive experiments with seven LLMs reveal their varying behaviors. |
HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-depth research on the specific capabilities needed by the RAG generation model is lacking, leading to inconsistent document quality and retrieval system imperfections. |
| Approach: | They propose that RAG models should possess three progressively hierarchical abilities: (1) Filtering: the ability to select relevant information; (2) Combination: the capability to combine semantic information across paragraphs; (3) RAG-specific reasoning: the capacity to further process external knowledge using internal knowledge. |
| Outcome: | Experiments show that the proposed method significantly improves the model’s open-book examination capability on datasets such as RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA. |
OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain (2025.emnlp-main)
Copied to clipboard
| Challenge: | a new benchmark for RAG is developed for the financial domain . omnidirectional and automatic benchmarks are difficult to build in vertical domains . |
| Approach: | They propose an omnidirectional and automatic RAG benchmark for the financial domain . they categorize RAG scenarios by task classes and 16 financial topics . |
| Outcome: | The proposed benchmark achieves an 87.47% acceptance ratio in human evaluations of generated instances. |
Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models (2025.findings-acl)
Copied to clipboard
Shuliang Liu, Xinze Li, Zhenghao Liu, Yukun Yan, Cheng Yang, Zheni Zeng, Zhiyuan Liu, Maosong Sun, Ge Yu
| Challenge: | Existing evaluation metrics cannot fairly evaluate the outputs of RAG models during training and evaluation. |
| Approach: | They propose a method which prompts LLMs to generate different judgments based on various combinations of judgment dimensions and utilizes the judge-consistency to evaluate these judgments. |
| Outcome: | The proposed method generates more accurate evaluations for RAG models across different RAG model and datasets. |
TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrievers are not perfect and often include irrelevant documents in the retrieved set. |
| Approach: | They propose to construct knowledge-grounded reasoning chains from retrieved documents to integrate supporting evidence into RAG models. |
| Outcome: | The proposed model achieves an average performance improvement of 14.03% on three multi-hop QA datasets. |
Eliciting Critical Reasoning in Retrieval-Augmented Generation via Contrastive Explanations (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies show that LLMs struggle to critically analyse RAG-based in-context information. |
| Approach: | They propose a framework that elicits critical arguments in RAG via contrastive explanations . they propose CRAG to retrieve relevant documents given a query and generate explanations that explicitly contrast relevance of passages to support the final answer. |
| Outcome: | The proposed framework improves state-of-the-art RAG models while requiring significantly fewer prompts and demonstrations and robust to perturbations in the retrieved documents. |
Who is in the Spotlight: The Hidden Bias Undermining Multimodal Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing RAG models are sensitive to the order in which evidence is presented, resulting in unstable performance and biased reasoning. |
| Approach: | They propose to quantify position bias in multimodal RAG systems by using position sensitivity index . they also develop a visualization framework to trace attention allocation patterns across decoder layers . |
| Outcome: | The proposed framework shows that multimodal interactions intensify position bias compared to unimodal settings and that this bias increases logarithmically with retrieval range. |
ConTReGen: Context-driven Tree-structured Retrieval for Open-domain Long-form Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing iterative retrieval-augmented generation approaches struggle to delve deeply into each facet of complex queries. |
| Approach: | They propose a framework that employs a tree-structured retrieval approach to enhance the depth and relevance of retrieved content. |
| Outcome: | The proposed framework outperforms state-of-the-art models on multiple datasets and a newly introduced dataset. |
Open-RAG: Enhanced Retrieval Augmented Reasoning with Open-Source Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to integrate Large Language Models with external knowledge suffer from limited reasoning capabilities, especially when using open-source LLMs. |
| Approach: | They propose a framework that transforms an arbitrary dense LLM into a parameter-efficient sparse mixture of experts (MoE) model capable of handling complex reasoning tasks. |
| Outcome: | The proposed framework transforms an arbitrary dense LLM into a parameter-efficient sparse mixture of experts (MoE) model capable of handling complex reasoning tasks, including both single- and multi-hop queries. |